adding session id to entries in access log

--000e0cd5c84c4933240474edc7fd
Content-Type: text/plain; charset=ISO-8859-1

We've just launched the first mod_perl site I've ever designed. It's all
going very well so far but I'm sure there are some things worth improving.
I wonder if anyone might have suggestions about this scenario:

I want to add the session id to the access log entries. This example:
http://perl.apache.org/docs/2.0/user/handlers/http.html#Perl LogHandler shows
how to write to a different logfile but I want to write to whatever would be
the normal access log for whatever VirtualHost it's in. We've only got a
PerlResponseHandler now but I think this should probably go in a
PerlLogHander. What's the best way to go about this?

We're very interested in tracking long-term user browsing behavior and so we
set one persistent cookie with a session key at each request if there's no
cookie or if the existing cookie is obsolete (user logged in, logged out,
more than 1 hour since last access, 12 since last visit, IP changed, or user
agent changed.) If, when a new session id is created there is an existing
(but obsolete) session cookie then the obsolete one is stored in the
sessions table as the previous session key.

I've tried to follow the philosophy that Randal Schwartz described in a
recent thread here - a cookie is just a serial number for a browser. By
rotating the cookies often we're hoping to avoid problems with stolen or
leaked sessions and by storing the previous session id (if there is one)
with every new session we're planning to be able to build a linked list of
session activity which we can correlate with specific users who log in at
any part of that linked list.

--000e0cd5c84c4933240474edc7fd
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable


We've just launched the first mod_perl site I've ever designed.=
=A0 It's all going very well so far but I'm sure there are some thi=
ngs worth improving.=A0 I wonder if anyone might have suggestions about thi=
s scenario:


I want to add the session id to the access log entries.=A0 This example=
: gHandler">http://perl.apache.org/docs/2.0/user/handlers/http .html#PerlLogHa=
ndler shows how to write to a different logfile but I want to write to =
whatever would be the normal access log for whatever VirtualHost it's i=
n. =A0 We've only got a PerlResponseHandler now but I think this should=
probably go in a PerlLogHander.=A0 What's the best way to go about thi=
s?


We're very interested in tracking long-term user browsing behavior =
and
so we set one persistent cookie with a session key at each request if there=
's no cookie or
if the existing cookie is obsolete (user logged in, logged out, more than 1=
hour since last access,
12 since last visit, IP changed, or user agent changed.)=A0 If, when a new =
session id is created there is an existing (but obsolete) session cookie th=
en the obsolete one is stored in the sessions table as the previous session=
key.




I've tried to follow the philosophy that Randal Schwartz described in a=
recent thread here - a cookie is just a serial number for a browser.=A0 By=
rotating the cookies often we're hoping to avoid problems with stolen =
or leaked sessions and by storing the previous session id (if there is one)=
with every new session we're planning to be able to build a linked lis=
t of session activity which we can correlate with specific users who log in=
at any part of that linked list.






--000e0cd5c84c4933240474edc7fd--

Re: adding session id to entries in access log

Douglas wrote:

> I want to add the session id to the access log entries.

We just added "%{session_id}C" as an additional field to
our CustomLog directive, and that worked fine for us.

http://httpd.apache.org/docs/2.0/mod/mod_log_config.html

--
Cosimo

Re: adding session id to entries in access log

>>>>> "Douglas" == Douglas Sims writes:

Douglas> I've tried to follow the philosophy that Randal Schwartz described in
Douglas> a recent thread here - a cookie is just a serial number for a
Douglas> browser. By rotating the cookies often we're hoping to avoid
Douglas> problems with stolen or leaked sessions and by storing the previous
Douglas> session id (if there is one) with every new session we're planning to
Douglas> be able to build a linked list of session activity which we can
Douglas> correlate with specific users who log in at any part of that linked
Douglas> list.

That's an interesting idea... brand the browser, but rotate it from time to
time, maintaining a list. Thanks for suggesting that... I'll have to explore
that in some future project.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095

Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Re: adding session id to entries in access log

--000e0cd6ac4203e3ec0475c415d0
Content-Type: text/plain; charset=ISO-8859-1

Thanks, Paul and Cosimo.

That module is just what I was looking for. Logging the note instead of the
cookie is probably better for what we're doing.






On Fri, Oct 2, 2009 at 6:56 AM, Paul Silevitch wrote:

> You can use apache's custom log (
> http://httpd.apache.org/docs/2.0/mod/mod_log_config.html) to log cookie
> values into the access logs:
>
> %...{Foobar}C The contents of cookie Foobar in the request sent to the
> server.
>
> The above will not log a value for the first request by a new visitor
> (since the cookie hasn't been set yet). Instead, create a note that gets
> set on every request in your handler and log that:
>
> %...{Foobar}n The contents of note Foobar from another module.
>
> HTH,
>
> Paul
>
>
>
> On Fri, Oct 2, 2009 at 2:21 AM, Douglas Sims wrote:
>
>>
>> We've just launched the first mod_perl site I've ever designed. It's all
>> going very well so far but I'm sure there are some things worth improving.
>> I wonder if anyone might have suggestions about this scenario:
>>
>> I want to add the session id to the access log entries. This example:
>> http://perl.apache.org/docs/2.0/user/handlers/http.html#Perl LogHandlershows how to write to a different logfile but I want to write to whatever
>> would be the normal access log for whatever VirtualHost it's in. We've
>> only got a PerlResponseHandler now but I think this should probably go in a
>> PerlLogHander. What's the best way to go about this?
>>
>> We're very interested in tracking long-term user browsing behavior and so
>> we set one persistent cookie with a session key at each request if there's
>> no cookie or if the existing cookie is obsolete (user logged in, logged out,
>> more than 1 hour since last access, 12 since last visit, IP changed, or user
>> agent changed.) If, when a new session id is created there is an existing
>> (but obsolete) session cookie then the obsolete one is stored in the
>> sessions table as the previous session key.
>>
>> I've tried to follow the philosophy that Randal Schwartz described in a
>> recent thread here - a cookie is just a serial number for a browser. By
>> rotating the cookies often we're hoping to avoid problems with stolen or
>> leaked sessions and by storing the previous session id (if there is one)
>> with every new session we're planning to be able to build a linked list of
>> session activity which we can correlate with specific users who log in at
>> any part of that linked list.
>>
>>
>>
>>
>

--000e0cd6ac4203e3ec0475c415d0
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks, Paul and Cosimo.

That module is just what I was looking for.=
=A0 Logging the note instead of the cookie is probably better for what we&#=
39;re doing.

Re: adding session id to entries in access log

--000e0cd72ae2dc89000475c4a8cc
Content-Type: text/plain; charset=ISO-8859-1

We're starting to get some data now and I'm seeing a few minor problems with
the implementation of the idea.

Here's one: if a request arrives with a cookie that is associated with a
different IP address then we create a new session entry and send a new
cookie. I'm noticing that some users have IP addreses that change very
frequently. This seems particularly likely with mobile devices. We're
going to have to tweak the algorithm a bit to track activity across those
requests - or use some attribute of the IP address instead of just the IP
address - to trigger rotation of the session identity.





On Sun, Oct 11, 2009 at 10:19 AM, Randal L. Schwartz
wrote:

> >>>>> "Douglas" == Douglas Sims writes:
>
> Douglas> I've tried to follow the philosophy that Randal Schwartz described
> in
> Douglas> a recent thread here - a cookie is just a serial number for a
> Douglas> browser. By rotating the cookies often we're hoping to avoid
> Douglas> problems with stolen or leaked sessions and by storing the
> previous
> Douglas> session id (if there is one) with every new session we're planning
> to
> Douglas> be able to build a linked list of session activity which we can
> Douglas> correlate with specific users who log in at any part of that
> linked
> Douglas> list.
>
> That's an interesting idea... brand the browser, but rotate it from time to
> time, maintaining a list. Thanks for suggesting that... I'll have to
> explore
> that in some future project.
>
> --
> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
>
> Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
> See http://methodsandmessages.vox.com/ for Smalltalk and Seaside
> discussion
>

--000e0cd72ae2dc89000475c4a8cc
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable


We're starting to get some data now and I'm seeing a few minor =
problems with the implementation of the idea.=A0

Here's one: if=
a request arrives with a cookie that is associated with a different IP add=
ress then we create a new session entry and send a new cookie.=A0 I'm n=
oticing that some users have IP addreses that change very frequently.=A0 Th=
is seems particularly likely with mobile devices.=A0 We're going to hav=
e to tweak the algorithm a bit to track activity across those requests - or=
use some attribute of the IP address instead of just the IP address - to t=
rigger rotation of the session identity.

Re: adding session id to entries in access log

--000e0cd6cef897ecfc0475c4f7d0
Content-Type: text/plain; charset=ISO-8859-1

Thanks! That's a good idea.

Just as an example, here are several IPs that seemed to be the same client.
The user agent, referer, etc. were all the same and the IPs resolve to the
same top-level domain.

205.228.12.236
205.228.12.151
205.228.12.254



On Mon, Oct 12, 2009 at 5:52 PM, Paul Silevitch wrote:

> Also, requests that go through a proxy can appear to come from different IP
> addresses from hit to hit (or visit to visit). Usually, proxies will put
> the real IP as the first IP in the X-Forwarded-For header.
>
> HTH,
>
> Paul
>
>
> On Mon, Oct 12, 2009 at 6:43 PM, Douglas Sims wrote:
>
>>
>> We're starting to get some data now and I'm seeing a few minor problems
>> with the implementation of the idea.
>>
>> Here's one: if a request arrives with a cookie that is associated with a
>> different IP address then we create a new session entry and send a new
>> cookie. I'm noticing that some users have IP addreses that change very
>> frequently. This seems particularly likely with mobile devices. We're
>> going to have to tweak the algorithm a bit to track activity across those
>> requests - or use some attribute of the IP address instead of just the IP
>> address - to trigger rotation of the session identity.
>>
>>
>>
>>
>>
>>
>> On Sun, Oct 11, 2009 at 10:19 AM, Randal L. Schwartz <
>> merlyn@stonehenge.com> wrote:
>>
>>> >>>>> "Douglas" == Douglas Sims writes:
>>>
>>> Douglas> I've tried to follow the philosophy that Randal Schwartz
>>> described in
>>> Douglas> a recent thread here - a cookie is just a serial number for a
>>> Douglas> browser. By rotating the cookies often we're hoping to avoid
>>> Douglas> problems with stolen or leaked sessions and by storing the
>>> previous
>>> Douglas> session id (if there is one) with every new session we're
>>> planning to
>>> Douglas> be able to build a linked list of session activity which we can
>>> Douglas> correlate with specific users who log in at any part of that
>>> linked
>>> Douglas> list.
>>>
>>> That's an interesting idea... brand the browser, but rotate it from time
>>> to
>>> time, maintaining a list. Thanks for suggesting that... I'll have to
>>> explore
>>> that in some future project.
>>>
>>> --
>>> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777
>>> 0095
>>>
>>> Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
>>> See http://methodsandmessages.vox.com/ for Smalltalk and Seaside
>>> discussion
>>>
>>
>>
>

--000e0cd6cef897ecfc0475c4f7d0
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks!=A0 That's a good idea.

Just as an example, here are seve=
ral IPs that seemed to be the same client.=A0 The user agent, referer, etc.=
were all the same and the IPs resolve to the same top-level domain.
>
205.228.12.236
205.228.12.151
205.228.12.254